Supporting Autonomic Management of Clouds: Service-Level-Agreement, Cloud Monitoring and Similarity Learning

Cloud computing has grown rapidly during the past few years and has become a fundamental paradigm in the Information Technology (IT) area. Clouds enable dynamic, scalable and rapid provision of services through a computer network, usually the Internet. However, managing and optimising clouds and their services in the presence of dynamism and heterogeneity is one of the major challenges faced by industry and academia. A prominent solution is resorting to selfmanagement as fostered by autonomic computing. Self-management requires knowledge about the system and the environment to enact the self-* properties. Nevertheless, the characteristics of cloud, such as large-scale and dynamism, hinder the knowledge discovery process. Moreover, cloud systems abstract the complexity of the infrastructure underlying the provided services to their customers, which obfuscates several details of the provided services and, thus, obstructs the effectiveness of autonomic managers. While a large body of work has been devoted to decisionmaking and autonomic management in the cloud domain, there is still a lack of adequate solutions for the provision of knowledge to these processes. In view of the lack of comprehensive solutions for the provision of knowledge to the autonomic management of clouds, we propose a theoretical and practical framework which addresses three major aspects of this process: (i) the definition of services’ provision through the specification of a formal language to define Service-Level-Agreements for the cloud domain; (ii) the collection and processing of information through an extensible knowledge discovery architecture to monitor autonomic clouds with support to the knowledge discovery process; and (iii) the knowledge discovery through a machine learning methodology to calculate the similarity among services, which can be employed for different purposes, e.g. service scheduling and anomalous behaviour detection. Finally, in a case study, we integrate the proposed solutions and show the benefits of this integration in a hybrid cloud test-bed

Um arcabouço de monitoramento e auto-proteção para nuvens privadas

Dissertação (mestrado) - Universidade Federal de Santa Catarina, Centro Tecnológico, Programa de Pós-Graduação em Ciência da Computação, Florianópolis, 2012Um dos novos desafios do paradigma de computação em nuvem é a administração efetiva destes sistemas e recursos devido, a sua heterogeneidade, escalabilidade e a falta de ferramentas adequadas. Consumo de energia, desempenho, provisão de recursos e segurança são somente alguns fatores relevantes no gerenciamento. Neste âmbito, a computação autônoma visa facilitar e automatizar este gerenciamento (gerenciamento sem intervenção humana) através de quatro propriedades: auto-otimização, auto-cura, auto-configuração e auto-proteção. O uso de computação autônoma em computação em nuvem, principalmente focando em nuvens privadas, foi pouco explorado até o momento. Este trabalho procura dar um dos primeiros passos para portar os princípios de computação autônoma para nuvens privadas com a definição de uma arquitetura para o monitoramento deste tipo de nuvem, uma das bases da computação autônoma. Esta também propõe o uso simplificado de umas das propriedades, a auto-proteção que se beneficia da base de monitoramento desenvolvida. Para validar esta proposta foi desenvolvido um arcabouço de código aberto e gratuito denominado PANOPTES. O Panoptes usa o paradigma multi-agente para o monitoramento efetivo, distribuído e escalável dos recursos físicos e virtuais da nuvem e, assim, fornece a base para a tomada correta de decisões. A interação com o administrador de sistemas e a sincronia com os objetivos da organização ocorre através da definição de políticas de alto nível. Dentre outras, as vantagens deste arcabouço são a facilidade de estender e adaptar o arcabouço para as próprias necessidades e a compatibilidade com os padrões em vigor. No decorrer do trabalho, os paradigmas supracitados e os pilares deste foram documentados para facilitar a sua compreensão, além de justificar as escolhas de cada parte importante na arquitetura, relacionar os padrões mais relevantes no desenvolvimento e uso destes. Por fim, um caso de uso é apresentado para validação da proposta

Service clustering for autonomic clouds using random forest

Managing and optimising cloud services is one of the main challenges faced by industry and academia. A possible solution is resorting to self-management, as fostered by autonomic computing. However, the abstraction layer provided by cloud computing obfuscates several details of the provided services, which, in turn, hinders the effectiveness of autonomic managers. Data-driven approaches, particularly those relying on service clustering based on machine learning techniques, can assist the autonomic management and support decisions concerning, for example, the scheduling and deployment of services. One aspect that complicates this approach is that the information provided by the monitoring contains both continuous (e.g. CPU load) and categorical (e.g. VM instance type) data. Current approaches treat this problem in a heuristic fashion. This paper, instead, proposes an approach, which uses all kinds of data and learns in a data-driven fashion the similarities and resource usage patterns among the services. In particular, we use an unsupervised formulation of the Random Forest algorithm to calculate similarities and provide them as input to a clustering algorithm. For the sake of efficiency and meeting the dynamism requirement of autonomic clouds, our methodology consists of two steps: (i) off-line clustering and (ii) on-line prediction. Using datasets from real-world clouds, we demonstrate the superiority of our solution with respect to others and validate the accuracy of the on-line prediction. Moreover, to show the applicability of our approach, we devise a service scheduler that uses the notion of similarity among services and evaluate it in a cloud test-bed

Supporting Autonomic Management of Clouds: Service Clustering with Random Forest

A promising solution for the management of services in clouds, as fostered by autonomic computing, is to resort to self-management. However, the obfuscation of underlying details of services in cloud computing, also due to privacy requirements, affects the effectiveness of autonomic managers. Data-driven approaches, in particular those relying on service clustering based on machine learning techniques, can assist the autonomic management and support decisions concerning, e.g., the scheduling and deployment of services. Unfortunately, applying such approaches is further complicated by the coexistence of different types of data within the information provided by the monitoring of cloud systems: both continuous (e.g., CPU load) and categorical (e.g., VM instance type) data are available. Current approaches deal with this problem in a heuristic fashion. In this paper, instead, we propose an approach that uses all types of data, and learns in a data-driven fashion the similarities and patterns among the services. More specifically, we design an unsupervised formulation of random forest to calculate service similarities and provide them as input to a clustering algorithm. For the sake of efficiency and to meet the dynamism requirement of autonomic clouds, our methodology consists of two steps: 1) off-line clustering and 2) on-line prediction. Using datasets from real-world clouds, we demonstrate the superiority of our solution with respect to others and validate the accuracy of the on-line prediction. Moreover, to show applicability of our approach, we devise a service scheduler that uses similarity among services, and evaluate its performance in a cloud test-bed using realistic data

Addressing Application Latency Requirements through Edge Scheduling

Abstract Latency-sensitive and data-intensive applications, such as IoT or mobile services, are leveraged by Edge computing, which extends the cloud ecosystem with distributed computational resources in proximity to data providers and consumers. This brings significant benefits in terms of lower latency and higher bandwidth. However, by definition, edge computing has limited resources with respect to cloud counterparts; thus, there exists a trade-off between proximity to users and resource utilization. Moreover, service availability is a significant concern at the edge of the network, where extensive support systems as in cloud data centers are not usually present. To overcome these limitations, we propose a score-based edge service scheduling algorithm that evaluates network, compute, and reliability capabilities of edge nodes. The algorithm outputs the maximum scoring mapping between resources and services with regard to four critical aspects of service quality. Our simulation-based experiments on live video streaming services demonstrate significant improvements in both network delay and service time. Moreover, we compare edge computing with cloud computing and content delivery networks within the context of latency-sensitive and data-intensive applications. The results suggest that our edge-based scheduling algorithm is a viable solution for high service quality and responsiveness in deploying such applications

Um Estudo Sobre Computação em Nuvem e o Monitoramento de Nuvens Privadas

TCC (graduação) - Universidade Federal de Santa Catarina. Centro Tecnológico. Curso de Sistemas de Informação.A tecnologia da informação está em constante mudança. Nos últimos anos vimos emergir um novo modelo que promete mudar o modo como usamos e entendemos a computação, usando Computação em Nuvem. Cloud computing ou Computação em Nuvem é um termo que vem sendo utilizado com muita frequência e sendo alvo de muita publicidade. Apesar desta tecnologia prometer revolucionar o mercado, muitas empresas não estão familiarizadas com o termo ou não conhecem seus benefícios. Esse desconhecimento adjunto ao fato de uma mídia apresentar computação em nuvem como uma tecnologia distante, faz com que uma mística seja criada em torno do tema. Consequentemente, grande parte dos profissionais de TI não sabem por onde começar e em geral, acreditam que computação em nuvem é um paradigma que se encaixa somente no contexto de grandes corporações. Neste trabalho é apresentada uma base teórica sobre essa tecnologia e tecnologias relacionadas. Também foi apresentado um estudo de caso usando a ferramenta Eucalyptus e desenvolvida uma solução de monitoramento focado em máquinas virtuais para suprir a falta de soluções atuais. Assim mostrando que a mística em torno da tecnologia é na verdade fruto da falta de informação, pois a tecnologia pode ser usada em muitos contextos, desde pequenas empresas até grandes corporações

Dynamic SLAs for Clouds

Part 1: Policies and PerformanceInternational audienceIn the Cloud domain, to guarantee adaptation to the needs of users and providers, Service-Level-Agreements (SLAs) would benefit from mechanisms to capture the dynamism of services. The existing SLA languages attempt to address this challenge by focusing on renegotiation of the agreement terms, which is a heavy-weight process, not really suitable for dealing with cloud dynamism. In this paper, we propose an extension of SLAC, a SLA language for clouds that we have recently defined, with a mechanism that enable dynamic modifications of the service agreement. We formally describe this extension, implement it in the SLAC framework and analyse the impacts of dynamic SLAs in some applications. The advantages of dynamic SLAs are demonstrated by comparing their effect with that of static SLA and of the “renegotiation” approach